Image region of interest encoding

ABSTRACT

Methods and apparatus, including computer program products, for compression include decomposing an image into wavelet transform coefficients, applying a first quantizer to the wavelet transform coefficients, and applying a second quantizer to a set of wavelet transform coefficients representing a background of the image. The method can include rounding the wavelet transform coefficients and rounding the wavelet transform coefficients can include rounding to a nearest integer value. The method can include multiplying the set rounded quantized background wavelet transform coefficients by the second quantizer and rounding all wavelet transform coefficients. Rounding all wavelet transform coefficients can include rounding to a nearest integer value.

BACKGROUND

The present invention relates to image region of interest encoding.

Image compression is minimizing the size in bytes of a graphics filewithout degrading the quality of the image to an unacceptable level. Thereduction in file size allows more images to be stored in a given amountof disk or memory space. It also reduces the time required for images tobe sent over the Internet or downloaded from Web pages.

There are several different standards that define the ways in whichimage files can be compressed. Three common standards, for example, arethe Joint Photographic Experts Group (JPEG) standard, the JPEG2000standard and the Graphics Interchange Format (GIF) standard.

The JPEG2000 digital image compression standard, for example, allows thecompression of an arbitrary region of interest (e.g., foreground) withinan image to be compressed at a higher quality than the remainder of theimage (e.g., background). The JPEG2000 digital image standard uses the“max shift” method in conjunction with a post compression ratedistortion (PCRD) optimization. In the max shift method, an image isfirst transformed into a wavelet domain and a subset of coefficientsthat contribute to a foreground are determined. The foregroundcoefficients are “left shifted” so that the modified foregroundcoefficients are larger than the largest possible backgroundcoefficient.

A decoder can now identify whether a coefficient belongs to theforeground or background by simply looking at its value. Thecoefficients of the foreground are now very large numbers and hence anerror occurring in these coefficients is penalized by a largedistortion. The PCRD optimization ensures that these foregroundcoefficients are included prior to background coefficients, whichensures that a large part of the bit budget is given to the encoding ofthe foreground coefficients.

SUMMARY

The present invention provides methods and apparatus, including computerprogram products, for image region of interest encoding.

In general, in one aspect, the invention features a method of encodingincluding decomposing an image into wavelet transform coefficients,quantizing the wavelet transform coefficients with a high resolutionscalar quantizer, quantizing a set of background wavelet transformcoefficients with a low resolution scalar quantizer, rounding the set ofquantized background wavelet transform coefficients, multiplying the setrounded quantized background wavelet transform coefficients by the lowresolution scalar quantizer, and rounding all wavelet transformcoefficients.

In embodiments, the method can include entropy encoding all wavelettransform coefficients. The method can include generating a preview ofthe encoded wavelet transform coefficients and/or signaling the highresolution scalar quantizer in a header.

Rounding all wavelet transform coefficients can include flooring to alargest integer smaller than a value of the quantized waveletcoefficient or choosing a smallest integer value larger than a value ofthe quantized wavelet coefficient.

In another aspect, the invention features a method of encoding includingdecomposing an image into wavelet transform coefficients, quantizing thewavelet transform coefficients with a high resolution scalar quantizer,rounding the wavelet transform coefficients, quantizing a set of roundedbackground wavelet transform coefficients with a low resolution scalarquantizer, rounding the set of quantized background wavelet transformcoefficients to a nearest integer value, and multiplying the set roundedquantized background wavelet transform coefficients by the lowresolution scalar quantizer.

In embodiments, the method can include entropy encoding all wavelettransform coefficients.

The method can include generating a preview of the encoded wavelettransform coefficients and/or signaling the high resolution scalarquantizer in a header.

Rounding the set of quantized background wavelet transform coefficientscan include rounding to a nearest integer value and rounding all wavelettransform coefficients can include rounding to a nearest integer value.

Rounding the set of quantized background wavelet transform coefficientsand rounding all wavelet transform coefficients can include flooring toa largest integer smaller than a value of the quantized waveletcoefficient.

Rounding the set of quantized background wavelet transform coefficientsand rounding all wavelet transform coefficients can include choosing asmallest integer value larger than a value of the quantized waveletcoefficient.

In another aspect, the invention features a method of compressionincluding decomposing an image into wavelet transform coefficients,applying a first quantizer to the wavelet transform coefficients, andapplying a second quantizer to a set of wavelet transform coefficientsrepresenting a background of the image.

In embodiments, the method can include rounding the wavelet transformcoefficients and rounding the wavelet transform coefficients can includerounding to a nearest integer value.

The method can include multiplying the set rounded quantized backgroundwavelet transform coefficients by the second quantizer and rounding allwavelet transform coefficients. Rounding all wavelet transformcoefficients can include rounding to a nearest integer value.

The method can include entropy encoding all wavelet transformcoefficients, generating a preview of the encoded wavelet transformcoefficients, and/or signaling the first quantizer in a header.

In another aspect, the invention features a system of encoding includinga means for decomposing an image into wavelet transform coefficients, ameans for quantizing the wavelet transform coefficients with a highresolution scalar quantizer, a means for quantizing a set of backgroundwavelet transform coefficients with a low resolution scalar quantizer, ameans for rounding the set of quantized background wavelet transformcoefficients, a means for multiplying the set rounded quantizedbackground wavelet transform coefficients by the low resolution scalarquantizer, and a means rounding all wavelet transform coefficients.

In embodiments, the system can include a means for entropy encoding allwavelet transform coefficients and/or a means for generating a previewof the encoded wavelet transform coefficients. The system can alsoinclude a means for signaling the high resolution scalar quantizer in aheader.

The means for rounding the set of quantized background wavelet transformcoefficients can include rounding to a nearest integer value. The meansfor rounding all wavelet transform coefficients can include rounding toa nearest integer value. The means for rounding all wavelet transformcoefficients can include flooring to a largest integer smaller than avalue of the quantized wavelet coefficient. The means for rounding allwavelet coefficients can include choosing a smallest integer valuelarger than a value of the quantized wavelet coefficient.

The invention can be implemented to realize one or more of the followingadvantages.

A user's choice of image quality for a region of interest (foreground)is mapped to a high resolution scalar quantizer and the image qualitychosen for a background is mapped to a low resolution (coarse) scalarquantizer. The quantizers for the foreground and the background can bechosen independently.

After wavelet transform, the transformed image includes severalsub-bands. The term quantization implies quantizing each coefficient ofeach sub-band with a quantizer; the quantizer can be different fordifferent sub-bands but is same for all coefficients within a sub-band.Thus, there is a set of high and coarse resolution quantizers, one foreach sub-band.

The image is transformed into the wavelet domain by applying the forwarddiscrete wavelet transform and the wavelet coefficients of each sub-bandthat contribute to the foreground are determined. All coefficients ofeach sub-band are then quantized, background as well as foreground, witha high resolution quantizer. The background is then quantized with thelow resolution quantizer, rounded to a nearest integer value andmultiplied back by the low resolution quantizer.

The modified wavelet coefficients are now encoded and the values of thehigh resolution quantizer are signaled in the encoded stream. Upondecoding, the foreground will appear with a better quality compared tothe background.

The process can encode different regions with more than two qualities.That is, one can have regions r1, r2, r3 . . . , with qualities q1, q2,q3 . . . . The process quantizes all wavelet transform coefficients withthe smallest quantizer (high resolution) and then each region ofinterest is individually subjected to quantization by the coarsequantizer for that region; rounding the coefficients to nearest integervalues and multiplying back by the coarse resolution quantizer.

The region of interest encoding process is not restricted to JPEG2000.The process can be used in any place where a wavelet based transform isperformed.

The quantized wavelet transform coefficients finally may or may not beentropy coded. Entropy encoding is not restricted to the JPEG2000entropy coding system. Entropy encoding can include, for example,Huffmann based systems, zip systems, flate, run length, and so forth.One need not perform any compression and only write the coefficients asthey are to a codestream.

The process can include generating a preview of the encoded image beforeperforming actual encoding. Thus, a user gets feedback on the quality ofthe foreground and background and can adjust the qualities to hissatisfaction.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features andadvantages of the invention will become apparent from the description,the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is block diagram.

FIG. 2 is a flow diagram.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

As shown in FIG. 1, the JPEG2000 standard includes encoding 10 anddecoding 12. We use the JPEG2000 standard as one example. However, theprocess described herein in not limited to the JPEG2000 standard, andcan be used in any place where a wavelet based transform is performed.

In encoding 10, a discrete wavelet transform 14 is first applied on thesource image data 16. The transform coefficients are then quantized 18and entropy coded 20, before forming the output codestream (bitstream)22. In decoding 12, the codestream 24 is first entropy-decoded 26,dequantized 28 and undergoes the inverse discrete wavelet transform 30to produce the reconstructed image 32. The JPEG2000 standard allowstiling of the image. Tiles are encoded independently.

As shown in FIG. 2, a process 100 of region of interest encodingincludes decomposing (102) an image into wavelet transform coefficients.A wavelet transform is a transformation to basis functions that arelocalized in both frequency and spatial domains. As basis functions oneuses wavelets. An advantage over a Fourier transform is the temporal (orspatial) locality of the base functions and the smaller complexity (O(N)instead of O(N log N) for the fast Fourier transform (where N is thedata size).

Process 100 includes quantizing (104) the wavelet transform coefficientswith a high resolution scalar quantizer. Quantization (104) is a processby which the coefficients are reduced in precision. This operation islossy, unless the quantization (104) is 1 and the coefficients areintegers, as produced by the reversible integer 5/3 wavelet. Forexample, each of the transform coefficients a_(b)(u,v) of a subband b isquantized to a value q_(b)(u,v) according to the formula:

${q_{b}\left( {u,v} \right)} = {{{sign}\left( {a_{b}\left( {u,v} \right)} \right)}\;\left\lfloor \frac{{a_{b}\left( {u,v} \right)}}{\Delta_{b}} \right\rfloor}$

The quantization Δ_(b) is represented relative to the dynamic rangeR_(b) of subband b, by the exponent ε_(b) and mantissa μ_(b) as:

$\Delta_{b} = {2^{R_{b} - z_{b}}\left( {1 + \frac{\mu_{b}}{2^{11}}} \right)}$

The dynamic range R_(b) depends on a number of bits used to representthe original image component and on the choice of the wavelet transform.All quantized transform coefficients are signed values even when theoriginal components are unsigned. These coefficients are expressed in asign-magnitude representation prior to coding.

Process 100 includes quantizing (106) a set of background wavelettransform coefficients, which have already been quantized (104) by ahigh resolution quantizer, with a low resolution (i.e., coarse) scalarquantizer and rounding (108) the set of quantized background wavelettransform coefficients.

The set of rounded quantized background wavelet transform coefficientsare multiplied (110) by the low resolution scalar quantizer and allwavelet transform coefficients are rounded (112).

Process 100 includes entropy encoding (114) all wavelet transformcoefficients. After entropy encoding (114) the high resolution scalarquantizer is signaled in a header.

Process 100 encodes a region of interest without performing a PCRDoptimization. Instead, a set of foreground wavelet transformcoefficients (e.g., a region of interest) and a set of backgroundwavelet transform coefficients are quantized by two differentquantizers. Since there are two independent quantizers, background andforeground qualities can be set independently. Process 100 does notintroduce extra bitplanes in the region of interest, and as PCRDoptimization is not involved, encoding is fast and memory constrained.

Images with regions of interest are smaller than images without regionsof interest because by quantizing the background with a coarsequantizer, rounding to nearest integer and multiplying back with thequantizer, many zeros are introduced in the lower significant bitplanesof the background. These zeros get compressed to a higher degree byentropy coding (114).

Process 100 can be understood more fully by an example. Consider thefollowing example set of wavelet transform coefficients belonging to aparticular sub-band from decomposing (102) an image:

10.30 5.70 11.20 9.80 20.50 21.40 9.00 6.10 4.20 10.70 8.30 12.60 13.0012.00 15.20 6.80 7.20 8.30 8.54 9.60 4.90

In this example, we designate columns 3, 4 and 5 as belonging to aregion of interest (also referred as a foreground), and columns 1, 2, 6and 7 as belonging to a background. Any region of interest can beselected by a user. In this example we arbitrarily select a value of alow resolution (coarse) scalar quantizer at 3.5 and a value of a highresolution scalar quantizer as 1.2.

After quantizing (104) the above wavelet transform coefficients with thehigh resolution scalar quantizer we obtain the following values for thewavelet transform coefficients:

8.58 4.75 9.33 8.16 17.08 17.83 7.50 5.08 3.50 8.92 6.92 10.5 10.8310.00 12.66 5.66 6.00 6.92 7.12 8.0 4.08

The set of background wavelet transform coefficients (i.e., coefficientscontained in columns 1, 2, 6 and 7) are quantized (106) with the lowresolution scalar quantizer to obtain the following results:

2.45 1.36 9.33 8.16 17.08 5.09 2.14 1.45 1.00 8.92 6.92 10.50 3.09 2.863.62 1.62 6.00 6.92 7.12 2.29 1.166

The set of background wavelet transform coefficients are rounded (108)to obtain the following:

2.00 1.00 9.33 8.16 17.08 5.00 2.00 1.00 1.00 8.92 6.92 10.50 3.00 3.004.00 2.00 6.00 6.92 7.12 2.00 1.00

The set of background wavelet transform coefficients above aremultiplied (110) by the low resolution scalar quantizer to obtain thefollowing:

7.00 3.50 9.33 8.16 17.08 17.50 7.00 3.50 3.50 8.92 6.92 10.50 10.5010.50 14.00 7.00 6.00 6.92 7.12 7.00 3.50

All wavelet transform coefficients are rounded (112):

7.00 4.00 9.00 8.00 17.00 18.00 7.00 4.00 4.00 9.00 7.00 11.00 11.0011.00 14.00 7.00 6.00 7.00 7.00 7.00 4.00

The above set of wavelet transform coefficients are entropy coded (114).The high resolution quantizer is signaled in a header.

The invention can be implemented in digital electronic circuitry, or incomputer hardware, firmware, software, or in combinations of them. Theinvention can be implemented as a computer program product, i.e., acomputer program tangibly embodied in an information carrier, e.g., in amachine-readable storage device or in a propagated signal, for executionby, or to control the operation of, data processing apparatus, e.g., aprogrammable processor, a computer, or multiple computers. A computerprogram can be written in any form of programming language, includingcompiled or interpreted languages, and it can be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment. Acomputer program can be deployed to be executed on one computer or onmultiple computers at one site or distributed across multiple sites andinterconnected by a communication network.

Method steps of the invention can be performed by one or moreprogrammable processors executing a computer program to performfunctions of the invention by operating on input data and generatingoutput. Method steps can also be performed by, and apparatus of theinvention can be implemented as, special purpose logic circuitry, e.g.,an FPGA (field programmable gate array) or an ASIC (application-specificintegrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for executing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto-optical disks, or optical disks. Information carrierssuitable for embodying computer program instructions and data includeall forms of non-volatile memory, including by way of examplesemiconductor memory devices, e.g., EPROM, EEPROM, and flash memorydevices; magnetic disks, e.g., internal hard disks or removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor andthe memory can be supplemented by, or incorporated in special purposelogic circuitry.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. Other features andadvantages of the invention will become apparent from the description,the drawings, and the claims.

1. A method of encoding comprising: decomposing an image into wavelettransform coefficients; quantizing the wavelet transform coefficientswith a high resolution scalar quantizer; quantizing a set of backgroundwavelet transform coefficients with a low resolution scalar quantizer;rounding the set of quantized background wavelet transform coefficients;multiplying the set of rounded quantized background wavelet transformcoefficients by the low resolution scalar quantizer; and rounding allwavelet transform coefficients; wherein quantizing the wavelet transformcoefficients is performed by one or more processors.
 2. The method ofclaim 1 further comprising entropy encoding all wavelet transformcoefficients.
 3. The method of claim 2 further comprising signaling thehigh resolution scalar quantizer in a header.
 4. The method of claim 1wherein rounding the set of quantized background wavelet transformcoefficients comprises rounding to a nearest integer value.
 5. Themethod of claim 1 wherein rounding all wavelet transform coefficientscomprises rounding to a nearest integer value.
 6. The method of claim 1wherein rounding all wavelet transform coefficients comprises flooringto a largest integer smaller than a value of the quantized waveletcoefficient.
 7. The method of claim 1 wherein rounding all waveletcoefficients comprises choosing a smallest integer value larger than avalue of the quantized wavelet coefficient.
 8. A method of encodingcomprising: decomposing an image into wavelet transform coefficients;quantizing the wavelet transform coefficients with a high resolutionscalar quantizer; rounding the wavelet transform coefficients;quantizing a set of rounded background wavelet transform coefficientswith a low resolution scalar quantizer; rounding the set of quantizedbackground wavelet transform coefficients to a nearest integer value;and multiplying the set of rounded quantized background wavelettransform coefficients by the low resolution scalar quantizer; whereinquantizing the wavelet transform coefficients is performed by one ormore processors.
 9. The method of claim 8 further comprising entropyencoding all wavelet transform coefficients.
 10. The method of claim 9further comprising signaling the high resolution scalar quantizer in aheader.
 11. The method of claim 8 wherein rounding the set of quantizedbackground wavelet transform coefficients comprises rounding to anearest integer value.
 12. The method of claim 8 wherein rounding allwavelet transform coefficients comprises rounding to a nearest integervalue.
 13. The method of claim 8 wherein rounding the set of quantizedbackground wavelet transform coefficients and rounding all wavelettransform coefficients comprises flooring to a largest integer smallerthan a value of the quantized wavelet coefficient.
 14. The method ofclaim 8 wherein rounding the set of quantized background wavelettransform coefficients and rounding all wavelet transform coefficientscomprises choosing a smallest integer value larger than a value of thequantized wavelet coefficient.
 15. A computer program, tangibly storedon a computer-readable storage device, for encoding an image, comprisinginstructions operable to cause a programmable processor to: decompose animage into wavelet transform coefficients; quantize the wavelettransform coefficients with a high resolution scalar quantizer; quantizea set of background wavelet transform coefficients with a low resolutionscalar quantizer; round the set of quantized background wavelettransform coefficients; multiply the set of rounded quantized backgroundwavelet transform coefficients by the low resolution scalar quantizer;and round all wavelet transform coefficients.
 16. The product of claim15 further comprising instructions operable to cause the programmableprocessor to entropy encode all wavelet transform coefficients.
 17. Theproduct of claim 16 further comprising instructions operable to causethe programmable processor to signal the high resolution scalarquantizer in a header.
 18. The product of claim 15 wherein rounding theset of quantized background wavelet transform coefficients comprisesrounding to a nearest integer value.
 19. The product of claim 15 whereinrounding all wavelet transform coefficients comprises rounding to anearest integer value.
 20. A system of encoding comprising: means fordecomposing an image into wavelet transform coefficients; means forquantizing the wavelet transform coefficients with a high resolutionscalar quantizer; means for quantizing a set of background wavelettransform coefficients with a low resolution scalar quantizer; means forrounding the set of quantized background wavelet transform coefficients;means for multiplying the set of rounded quantized background wavelettransform coefficients by the low resolution scalar quantizer; and meansfor rounding all wavelet transform coefficients.
 21. The system of claim20 further comprising means for entropy encoding all wavelet transformcoefficients.
 22. The system of claim 21 further comprising means forgenerating a preview of the encoded wavelet transform coefficients. 23.The system of claim 21 further comprising means for signaling the highresolution scalar quantizer in a header.
 24. The system of claim 20wherein means for rounding the set of quantized background wavelettransform coefficients comprises rounding to a nearest integer value.25. The system of claim 20 wherein means for rounding all wavelettransform coefficients comprises rounding to a nearest integer value.26. The system of claim 20 wherein means for rounding all wavelettransform coefficients comprises flooring to a largest integer smallerthan a value of the quantized wavelet coefficient.
 27. The system ofclaim 20 wherein means for rounding all wavelet coefficients compriseschoosing a smallest integer value larger than a value of the quantizedwavelet coefficient.